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EXECUTIVE  SUMMARY 


This  document  investigates  the  issue  of  fidelity  and  the  role  it 
plays  in  choosing  the  appropriate  simulation  device  flight 

deck  htiman  factors  research.  Fidelity  is  concerned  with  the 
degree  to  which  a  flight  simulator  matches  the  characteristics  of 
the  real  airplane.  An  extensive  search  of  the  scientific 
literature  provided  the  most  current  information  related  to 
simulator  fidelity  and  how  to  determine  what  levels  of  fidelity 
are  necessary  for  simulated  experimental  research.  The  purpose 
of  this  report  is  to  provide  basis  material  to  guide  the  Federal 
Aviation  Administration  (FAA)  in  stipulating  the  simulator^ 
sophistication  (level  of  fidelity)  reguired  to  conduct  various 
types  of  flight  deck  experimental  research. 

This  analysis  defines  a  research  simulation  device  and  identifies 
the  three  general  categories  of  aircraft  simulation  devices : 

(1)  airplane  simulators,  (2)  airplane  flight  training  devices, _ 
and  (3)  computer-based  simulation  devices.  The  use  of  simulation 
is  an  extremely  important  resource  for  most  aviation  human 
factors  research  and  development  programs .  This  importance  is 
due  to  a  variety  of  factors,  including  cost  and  time  savings  that 
can  be  realized,  the  ability  to  reproduce  and  examine  situations 
that  would  be  unsafe  using  actual  equipment,  and  the  control  and 
measurement  of  human-machine  performance.  Simulation  provides  an 
early  opportunity  to  bring  experienced  flight  crews  into  the ^ 
aviation  human  factors  design  process  to  assess  and  insure,  in 
paif't icular ,  proper  man/machine  interfaces  and  workload  levels. 

Additional  impetus  has  been  provided  by  the  tremendous 
technological  advances  in  computer  software  and  hardware 
capabilities  being  incorporated  into  these  devices.  These  robust 
capabilities  are  being  obtained  at  the  expense  of  increased 
developmental,  operational,  and  maintenance  costs.  This  increased 
cost  has  caused  flight  simulator  users  to  look  closely  at  the 
simulator  requirements  necessary  to  effectively  perform  their 
tasks,  and  then  invest  enough  resources  to  obtain  a  simulation 
device  that  meets  their  specified  needs.  Failure  to  properly 
determine  these  simulator  requirements  can  result  in  (1) 
unsatisfactory  results  due  to  lack  of  realism,  or  (2) 
satisfactory  results  but  at  premium  cost  (suboptimization) . 

Before  deciding  what  simulation  requirements  are  necessary  for  a 
specific  need,  researchers  need  to  understand  simulator 
effectiveness  and  the  concept  used  to  describe  it,  fidelity.  The 
concept  of  fidelity  relates  to  the  degree  to  which  the 
cjiaracteris t ics  of  a  flight  simulator  match  those  of  the  actual 
aiifplane .  Simulator  fidelity  has  been  discussed  and  studied  for 
over  30  years,  and  there  is  still  no  agreed  upon  single 
definition.  During  this  time,  the  term  has  been  used  in  a 
variety  of  ways  and  to  refer  to  many  different  aspects  of 
simulation.  A  representative  sample  of  the  different  kinds  of 
fidelity  includes  such  things  as equipment _ fidelity, 
environmental  fidelity,  psychological  fidelity,  task  fidelity. 
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physical  fidelity,  and  functional  fidelity.  A  common  thread, 
however,  is  that  together  these  definitions  imply  at  least  two 
major  features  or  dimensions  along  which  these  simulated  devices 
may  differ  from  actual  equipment. 

These  two  features  represent  the  division  of  a  simulator  into  two 
classes  depending  on  the  nature  of  the  cues  they  provide. 

1.  Equipment  cues  provide  a  duplication _ of  the  appearance  and 
feel  of  the  operational  equipment  (the  aircraft),  i-®-' 
static  and  internal  dynamic  characteristics  such  as  the  size 
shape,  location,  and  color  of  controls  and  displays,  including 
controller  force  and  displacement  characteristics . 

2.  Environment  cues  provide  a  duplication _ of  the  enviro^ent  and 
motion  through  the  environment.  Fidelity  is  then  a 

the  degree  to  which  the  equipment  and  environmental  cues  relate 

to  those  of  the  real  airplane. 

Amona  the  two  major  categories  of  simulators,  training  and 
research,  there  is  a  tradeoff  between  equipment  and  environmental 
cue  fidelity  required.  The  physical  correspondence  between 
simulator  and  airplane  in  terms  of  cockpit  layout,  flight 
instruments,  controls,  etc.,  for  training 

result  in  high  equipment  cue  fidelity  to  advocate  a  high  degree 
of  transfer  of  training  to  the  operational  environment.  On  the 
other  hand,  research  and  development  simulators  should  place  more 
emphasis  on  high  environmental  cue  fidelity.  Enyiro^ental  cues, 
by^definition,  provide  duplication  of  the  operational  aircraft 
eLironment  and  motion,  and  thus  will  result  in  a  higher  degree 
of  realism  being  experienced  by  the  subjects.  This  . 

realism  will  result  in  subject  performance  more  closely  matching 
that  which  would  occur  in  the  real  world. 

Once  the  concept  of  fidelity  and  its  effect  on  simulation  is 
understood,  requirements  for  simulation  experimental  research  can 
be  examined.  The  essential  feature  of  simulator  experimenta 
investiqations  is  to  introduce  the  pilots  into  a  closed  loop 
control  situation,  so  that  account  is  taken  of  their  cgabilities 
and  limitations  regarding  the  performance  or  behavior  being 
evaluated.  The  expectation  is  that 

experimental  conditions,  behavior  in  the  simulator  will  match 
thSir  behavior  in  the  flight  situation.  Hence,  the  primary  goal 
for  a  flight  simulation  researcher  is  to  produce  experiment 
conditions  that  elicit  behavior  that  would  occur  under  similar 
circumstances  in  the  real  world.  However,  regardless  of  the 
level  of  fidelity  and  how  accurately  it  was  determined  to  meet 
the  goals  of  the  research,  no  simulator  evaluation  can  completely 
duplicate  the  experience  in  the  real  world.  Therefore,  any 
results  should  be  verified  operationally  before  definite 
conclusions  can  be  made. 

When  planning  human  factors  research,  the  required 
characteristics  and  features  of  the  research  vehicle 


viii 


proiniiient  issuGS  of  consid.6ira.tion  in  thG  goal  of  choosing  thG 
right  simulator  for  the  planned  research  investigation.  Two 
principal  factors  in  determining  the  choice  of  research  vehicle 
are  (1)  the  type  of  research  required  by  the  problem,  and  (2) 
knowledge  of  the  factors  that  influence  the  behavioral  processes 
of  interest.  The  type  of  research  dictates  the  level  of  .  _ 

representation  that  the  research  vehicle  must  have.  Knowledge  of 
the  factors  which  influence  the  behavioral  processes  determine 
how  comprehensively  the  research  vehicle  must  represent  an 
operational  system  and  its  associated  situational  condition. 
Together,  these  features  are  commonly  thought  of  as  the  fidelity 
of  the  research  vehicle. 


Simulation  researchers  are  faced  with  the  many  problems, 
considerations,  and  conflicting  issues  relating  to  the 
determination  of  fidelity  requirements  for  a  simulation  research 
effort.  Fidelity  is  a  multivariant  construct  with  no  consensus 
among  researchers  of  a  single  index  of  measurement  or  definition. 
Given  the  degree  of  differences  and  difficulties  relating  to 
simulator  fidelity,  specific  guidelines  for  various  types  of 
experimental  research  are  not  presently  available. _  However, 
general  conclusions  can  be  made  regarding  how  varying  levels  of 
fidelity  can  effect  the  outcome  of  different  types  of 
experiments.  Experimental  research  studies  performed  using  a 
simulation  device  generally  fall  into  two  categories,  full^ 
mission  and  part-task.  Full-mission  studies  examine  behavior  in 
the  full  context  of  the  aviation  environment,  while  part-task 
studies  concentrate  on  the  behavior  relating  to  a  specific  tas 
or  function.  Within  these  two  categories  there  are  distinct 
study  types  that  are  distinguished  by  the  complexity  of  their 
objectives  or  the  type  of  simulation  device  used  for 
investigation. 

The  basic  idea  of  full-mission  relates  to  performing  a  research 
study  with  the  most  realistic  simulation  possible.  It  includes 
the  aircraft  cockpit,  visual  and  motion  cues,  aircraft  flight 
dynamics,  all  of  the  aircraft  subsystems,  the  flight _ environment 
(including  air  traffic  control,  weather,  and  other  air  or  ground 
vehicular  traffic) ,  the  cabin  crew,  and  all  ancillary  flight 
services  (such  as  dispatch,  ramp  passenger  services,  and 
maintenance) .  Given  these  characteristics  of  full-mission 
simulation,  it  can  be  accurately  stated  that  only  airplane 
simulators  should  be  used  for  full-mission  simulation^ 
applications.  The  airplane  simulators  are  the  only _ simulation 
devices  that  specify  visual  and  motion  characteristics  by 
definition,  and  these  capabilities  are  definitely  necessary  when 
conducting  full-mission  simulation  research. _  A  full-mission 
simulation  can  be  used  to  investigate  cognitive  tasks  in  t  e 
context  of  the  multitask,  complex  operation  of  flying  an 
airplane.  Also,  much  of  the  research  performed  using  full 
mission  simulation  focuses  on  cockpit  instrumentation,  crew 
procedures,  and  workload  measurement. 
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A  main  fidelity  issue  of  concern  in  full-mission  simulation  is 
the  pilot  subjects'  desire  for  scenario  fidelity.  Pilots 
generally  do  not  accept  deviations  from  operational  practice 
unless  it  is  specified  at  the  beginning  of  the  simulation. 

Pilots  experiencing  negative  user  acceptance  cues  are  more  likely 
not  to  elicit  the  same  behavior  as  they  might  in  the  real  world. 
Hence,  for  full-mission  simulation  research  studies,  the  higher 
the  level  of  fidelity  the  more  likely  their  behavior  will 
represent  that  of  the  operational  environment  and  therefore  be 
generalizable  to  the  real  world. 

The  concept  of  a  part-task  study  is  to  investigate  a  performance 
measure  in  response  to  a  specific  manipulated  task  or  function. 
These  types  of  investigations  isolate  a  single  critical  function 
for  evaluation  in  terms  of  pilot  behavior.  The  benefit  of  part- 
task  simulation  is  derived  from  the  view  that  the  evaluation  of 
smaller  component  tasks  is  more  acceptable  for  experimental 
testing  and  statistical  analysis  and  more  objective  information 
regarding  performance  can  be  obtained.  Basically,  studies 
looking  at  human  performance  on  a  specific  individual  task 
(reaction  time,  accuracy,  etc.),  or  functional  problems  inherent 
to  a  task  or  condition  can  be  evaluated  using  part-task 
simulation.  However,  what  part- task  simulation  may  gain  in 
experimental  control  it  lacks  in  external  validity,  i.e., 
accurate  representation  of  the  real  world. 

Part-task  simulations  can  utilize  a  wide  variety  of  simulation 
devices  such  as:  microcomputer  simulation  devices,  low-fidelity 
desktop  training  devices,  and  high  fidelity  airplane  flight 
training  devices.  This  wide  range  of  fidelity  devices 
complicates  the  issue  of  determining  specific  requirements  for 
part-task  simulation  fidelity.  The  utilization  of  too  much 
fidelity  can  result  in  unwanted  variance  associated  to  the 
behavior  being  examined.  On  the  other  hand,  if  the  simulation 
does  not  represent  the  context  in  which  the  specific  task  is  to 
be  investigated,  due  to  a  low  level  of  fidelity,  the  behavior 
examined  may  not  be  exactly  that  for  which  the  research  was 
intended.  Hence,  fidelity  requirements  for  part-task  simulation 
studies  cannot  be  determined  in  general .  The  requirements  must 
be  determined  on  a  case-by-case  basis  depending  on  the  objectives 
of  the  research. 

In  conclusion,  simulation  fidelity  is  an  obscure  concept  that  is 
being  thrust  onto  the  simulation  community  as  a  way  to  measure  a 
simulation  device's  effectiveness  for  human  factors  research. 

To  date,  no  consensus  on  just  what  exactly  fidelity  is,  or  how  it 
affects  simulation  research  efforts  has  been  reached. 
Additionally,  the  amount  of  research  available  that  investigates 
fidelity  requirements  for  research  simulators  is  not  abundant, 
and  no  real  guidelines  have  ever  been  agreed  upon  by  simulator 
researchers.  To  reiterate,  simulation  fidelity  requirements  are 
dependent  specifically  on  what  the  simulator  is  to  be  used  for . 

A  well-designed  research  simulation  project  is  cost-effective 
when  compared  to  most  other  ways  of  achieving  the  same 


objectives,  such  as  flight  test.  However,  as  the  use  of 
simulation  for  research  increases,  more  specific  guidelines  and 
requirements  for  fidelity  are  necessary  to  ensure  that  the 
simulation  devices  are  being  used  effectively  to  meet  the 
objectives  of  the  specified  research.  Recommendations  for  future 
research  to  provide  more  specific  requirements  for  human  factors 
simulation  research  are  presented  at  the  end  of  the  document. 
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INTRODUCTION. 


Flight  simulation  is  a  vital  part  of  aeronautical  research.  This 
research  is  conducted  on  a  broad  front,  both  at  research 
establishments  and  in  industry.  Simulation  ranges  from 
comprehensive  representations  of  the  operational  equipment  and 
environment  in  support  of  full-mission  performance  to  modest 
simulations  involving  a  single  item  of  equipment  or  part  task. 
Simulation  allows  researchers  to  combine  real-world  hardware, 
environmental  conditions,  and  task  demands  with  the  ability  to 
control  events  and  conditions.  Also,  research  simulators  allow 
designers  to  explore  the  implications  of  different  design  options 
without  having  to  incur  the  expense  and  delay  arising  from 
building  and  testing  a  range  of  prototypes.  In  addition,  flight 
simulation  research  has  provided  a  means  of  evaluating  the  likely 
behavior  and  consequences  arising  from  abnormal  operating 
configurations  without  jeopardizing  the  safety  of  the  flight 
crews.  Ultimately,  the  use  of  simulation  will  result  in  data 
being  collected  faster  and  more  economically  than  in  the  real 
world. 

The  use  of  flight  simulation  in  research  has  extended 
considerably  in  the  last  20  years,  as  equipment  improvements  have 
become  available.  As  this  growth  continues,  there  are  many 
issues  that  are  being  debated  regarding  the  use  of  simulation  for 
experimental  research  and  development.  One  of  the  major  issues 
surrounding  human  factors  flight  simulation  research  involves 
determining  what  level  of  simulator  fidelity  is  necessary  to 
provide  satisfactory  experimental  results. 

One  group  among  today's  heavy  users  of  flight  simulators 
continues  to  strive  for  as  much  realism  as  possible,  i.e.,  (high 
fidelity) .  The  belief  here  is  that  the  more  realistic  a 
simulation  is  perceived  by  the  pilots,  the  more  their  behavior 
mimics  that  in  the  operational  environment.  Technology 
advancements  have  made  the  goal  of  a  motion-based  simulator  with 
a  wraparound  visual  system  and  the  exact  duplication  of  every 
detail  of  the  cockpit  a  reality.  The  high  degree  of  cockpit 
similarity  in  this  type  of  simulator  conveys  a  high  degree  of 
face  fidelity,  or  overall  representation  of  real-world 
characteristics.  This  face  fidelity  has  played  a  major  role  in 
gaining  acceptance  from  professional  pilots  for  using  simulators 
to  conduct  experimental  research.  Be  this  as  it  may,  with  the 
ever-increasing  use  of  flight  simulators  and  their  expanding 
costs,  other  groups  in  the  simulation  research  community  have 
long  supported  the  use  of  lower  levels  of  fidelity  depending  on 
the  goal  of  the  simulation. 

The  work  described  herein  analyzes  simulator  fidelity  issues  and 
discusses  fidelity  requirements  for  human  factors  simulation 
research  studies.  An  extensive  literature  review  was  performed _ 
on  the  general  topic  of  research  simulator  fidelity  issues .  This 
topic  was  investigated  to  provide  guidance  information  for  the 
Federal  Aviation  Administration  (FAA)  to  consider  when 
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determining  fidelity  requirements  for  specific  types  of  human 
factors  experimental  research  studies.  The  general  concept  of 
fidelity  will  be  introduced  and  issues  concerning  various  levels 
of  fidelity  in  simulation  research  are  summarized. 


1.1  PURPOSE /SCOPE. 

The  information  contained  herein  provides  a  handbook  to  guide  the 
FAA  in  selecting  experimental  apparatus  as  a  function  o  a  given 
research  study.  This  handbook  will  support  the  FAA's  future 
flight  deck  data  link  human  factors  research  program  that 
utilized  their  established  Cockpit  Simulator  Network  (CSN) . 
Material  found  in  this  handbook  discusses  how  to  utilize 
resources  (simulators)  in  the  most  efficient  (technically)  and 
cost  effective  way,  given  that  as  fidelity  increases,  cos 
increases.  Issues  regarding  necessary  simulator  fidelity 
requirements  for  conducting  human  factors  research  are  addressed. 

1.2  ORGANIZATION  OF  THE  REPORT. 

The  report  will  begin  with  a  brief  introduction  to  the  various 
types  of  research  simulation  devices  and  proceeds  to  classi  y 
them  into  three  major  categories  based  on  their  characteristics. 
The  categories  are  (1)  Airplane  Simulator,  (2)  Airplane  Flig 
Training  Device,  and  (3)  Computer-Based  Simulation  Device.  , 

the  term  "Fidelity"  is  defined  with  regards  to  simulation  and  the 
individual  components  that  drive  varying  levels  of  fidelity  are 
discussed  in  detail.  The  following  section  examines  research 
simulator  fidelity  requirements  in  general,  and  discusses  tne 
conflicting  trains  of  thought  in  the  simulation  community 
regarding  the  level  of  fidelity  necessary  to  obtain  significant 
results.  The  fourth  section  examines  general  fidelity 
requirements  for  research  simulators  and  discusses  the  advantages 
and  corresponding  disadvantages  associated  with  different  levels 
of  simulator  fidelity. 

The  last  major  section  (Human  Factors  Research  Fidelity  Concerns) 
looks  at  the  fidelity  characteristics  of  the _ two  main  t^es  ot 
human  factors  simulation  research,  full-mission  and  part  task, 
and  examines  what  type  of  performance  measures  should  be 
evaluated  in  each.  Also,  the  type  of  simulation  devices  to  be 
utilized  for  conducting  full-mission  and  part-task  simulation 
research  are  discussed.  Furthermore,  recommendations  for 
specific  types  of  simulation  devices  that  can  be  used  for  various 
research  objectives  is  provided  in  this  section.  Finally,  the 
last  section  of  the  report  will  provide  recommendations  for 
future  research  to  further  investigate  the  construct  of  fidelity 
and  its  significance  and  influence  on  conducting  human  factors 
simulation  research  studies. 

2  ■  SIMTJLATFD  EXPFRIMENTAL  RESEARCH  DEVICES  . 

The  use  of  simulation  is  an  extremely  important  resource  for  most 
aviation  human  factors  research  and  development  programs. 
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simulation  provides  an  early  opportunity  to  bring  experienced 
flight  crews  into  the  aviation  human  factors  design  process  to 
assess  and  insure,  in  particular,  proper  man/machine  interfaces 
and  workload  levels .  In  this  manner  these  flight  crews  are  able 
to  contribute  to  control  system  design  and  flight  deck  layout  and 
to  the  integration/operation  of  complex  subsystems,  like  an 
integrated  Flight  Management  System  (FMS) .  Also,  research  and 
development  simulators  can  provide  a  means  of  evaluating  the 
likely  behavior  and  consequences  arising  from  abnormal  operating 
configurations. 

In  terms  of  equipment,  research  and  development  simulated  devices 
run  the  full  spectrum  from  computer-based  flight  simulators,  to 
desktop  part-task  simulators,  and  ultimately  to  full-mission, 

6  Degrees  of  Freedom  (DOF)  flight  simulators  with  full  out-the- 
window  visual  systems.  The  ever-increasing  emphasis  on  the 
utility  of  research  and  development  devices  is  evident  by  the 
significant  increase  of  this  type  of  equipment  appearing  across 
the  world  in  government  laboratories,  universities,  and  civilian 
industries . 

Aviation  simulation  used  for  research  includes  numerous  different 
typ0s  of  devices  distinguished  by  technical  capabilities  and 
capital  investment  required.  For  this  analysis  of  fidelity 
requirements,  three  categories  will  be  defined  to  encompass  all 
aviation  simulated  research  devices:  (1)  Airplane  Simulator,  (2) 
Airplane  Flight  Training  Device,  and  (3)  Computer-Based 
Simulator.  Throughout  the  remainder  of  this  document,  the 
general  term  "simulator"  will  be  used  to  refer  to  all  three 
categories  of  simulation  devices,  unless  otherwise  specified. 
Within  these  broad  categories  of  devices  there  is  further 
distinction  based  on  their  associated  level  of  objective  fidelity 
(physical  realism  to  the  real  world) .  For  example,  the  FAA  has 
published  two  separate  FAA  Advisory  Circulars  (AC) ,  FAA-AC-120- 
40B  and  FAA-AC-120-45A,  that  contain  training  qualification 
requirements  for  airplane  simulators  and  airplane  flight  training 
devices,  respectively,  and  classify  these  devices  in  terms  of 
objective  fidelity.  Simulator  classification  by  objective 
fidelity  sets  a  basis  from  which  the _ training _ community  can 
identify  the  specific  simulation  device  that  is  optimized  for 
their  needs  (Prasad,  et  al . ,  1991) .  A  definition  and  detailed 
description  of  the  various  levels  of  simulation  capability  for 
each  category  of  simulation  device  is  described  below. 

2.1  AIRPT.ANE  SIMULATOR. 

An  airplane  simulator  is  a  full  size  replica  of  a  specific  type 
of  make,  model,  and  series  airplane  cockpit  (e.g.,  Boeing  727-200 
or  MD-80-20) ,  including  the  assemblage  of  equipment  and  computer 
software  programs  necessary  to  represent  the  airplane  in  ground 
and  flight  operations,  a  visual  system  providing  an  out-of-the- 
cockpit  view,  a  force  (motion)  cueing  system  which  provides  cues 
at  least  equivalent  to  that  of  a  3  DOF  motion  system;  and  is  in 
compliance  with  the  minimum  standards  specified  in  FAA  Advisory 
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culair  120-40B,  as  ainanded  in  July  1991.  A  functional 
description  for  each  of  the  four  levels  of  airplane  simulators 
are  given  below.  Differences  in  each  simulator's  description,  as 
the  level  of  sophistication  increases  from  Level  A  to  Level  D, 
are  listed  below. 

Level  A  Flight  Simulator 

Functional  Description 

a.  Systems  representations,  switches,  and  controls  which  are 
required  by  the  type . 

b.  Design  of  the  aircraft  and  by  the  user's  approved 
training  program. 

c.  Systems  which  respond  appropriately  and  accurately  to  the 
switches  and  controls  of  the  aircraft  being  simulated. 

d.  Full-scale  replication  of  the  cockpit  of  the  aircraft 
being  simulated. 

e.  Correct  simulation  of  the  aerodynamic  characteristics  of 
the  aircraft  being  simulated. 

f.  Correct  simulation  of  the  effects  of  the  selected 
environmental  conditions  which  the  simulated  aircraft  might 
encounter . 

g.  Control  forces  and  travel  which  correspond  to  the 
aircraft . 

h.  Instructor  controls  and  seat. 

i.  At  least  a  night  visual  system  with  at  least  a  45° 
horizontal  by  30°  vertical  field  of  view  for  each  pilot  station. 

j .  A  motion  system  with  at  least  3  DOF. 

Level  B  Flight  Simulator 

Functional  Description 

a.  Systems  representations,  switches,  and  controls  which  are 
required  by  the  type  design  of  the  aircraft  and  by  the  user  s 
approved  training  program. 

b.  Systems  which  respond  appropriately  and  accurately  to  the 
switches  and  controls  of  the  aircraft  being  simulated. 

c.  Full-scale  replication  of  the  cockpit  of  the  aircraft 
being  simulated. 
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d.  Correct  simulation  of  the  aerodynamic  characteristics 
including  ground  effect,  and  ground  dynamic  characteristics  of 
the  aircraft  being  simulated. 

e.  Correct  simulation  of  the  effects  of  the  selected 
environmental  conditions  which  the  simulated  aircraft  might 
encounter . 

f .  Control  forces  and  travel  which  correspond  to  the 
aircraft . 

g.  Instructor  controls  and  seat. 

h.  At  least  a  night  visual  system  with  at  least  a  45° 
horizontal  by  30°  vertical  field  of  view  for  each  pilot  station. 

I.  A  motion  system  with  at  least  3  DOF. 

Level  C  Flight  Simulator 
Functional  Description 

a.  Systems  representations,  switches,  and  controls  which  are 
required  by  the  type . 

b.  Design  of  the  aircraft  and  by  the  user's  approved 
training  program. 

c .  Systems  which  respond  appropriately  and  accurately  to  the 
switches  and  controls  of  the  aircraft  being  simulated. 

d.  Full-scale  replication  of  the  cockpit  of  the  aircraft 
being  simulated. 

e.  Correct  simulation  of  the  aerodynamic  characteristics 
including  ground  effect,  and  ground  dynamic  characteristics  of 
the  aircraft  being  simulated. 

f.  Correct  simulation  of  the  effects  of  the  selected 
environmental  conditions  which  the  simulated  aircraft  might 
encounter . 

g.  Control  forces  and  travel  which  correspond  to  the 
aircraft . 

h.  Instructor  controls  and  seat. 

i .  At  least  a  night  and  dusk  visual  system  with  at  least  a 
75°  horizontal  by  30°  vertical  field  of  view  for  each  pilot 
station . 

j.  A  motion  system  with  at  least  6  DOF. 
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Level  D  Flight  Simulator 
Functional  Description 


a.  Systems  representations,  switches,  and  controls  which  are 
required  by  the  type  design  of  the  aircraft  and  by  the  user  s 
approved  training  program. 

b.  Systems  which  respond  appropriately  and  accurately  to  the 
switches  and  controls  of  the  aircraft  being  simulated. 

c.  Full-scale  replication  of  the  cockpit  of  the  aircraft 
being  simulated. 

d.  Correct  simulation  of  the  aerodynamic  characteristics 
including  ground  effect,  and  ground  dynamic  characteristics  of 
the  aircraft  being  simulated. 

e.  Correct  simulation  of  selected  environmental  affected 
aerodynamic  and  ground  dynamic  characteristics  of  the  aircraft 
being  simulated  considering  the  full  range  of  its  flight  envelope 
in  all  approved  configurations. 

d.  Control  forces,  dynamic,  and  travel  which  correspond  to 
the  aircraft  instructor  controls  and  seat. 

e.  A  daylight,  dusk,  and  night  visual  system  with  at  least  a 
75°  horizontal  by  3  0°  vertical  field  of  view  for  each  pilot 
station. 

f.  A  motion  system  with  at  least  6  DOF. 

2.2  AIRPLANE  FLIGHT  TRAINING  DEVICE. 

An  airplane  flight  training  device  is  a  full-scale  replica  of  an 
airplane's  instruments,  equipment,  panels,  and  controls  in  an 
open  flight  deck  area  or  an  enclosed  airplane  cockpit.  Flight 
training  devices  also  include  the  assemblage  of  equipment  and 
computer  software  programs  necessary  to  represent  the  airplane  in 
ground  and  flight  conditions  to  the  extent  of  the  systems 
installed  in  the  device.  A  force  (motion)  cueing  or  visual  _ 
system  is  not  required.  It  must  meet  the  criteria  outlined  in 
FAA  Advisory  Circular,  FAA-AC-120-45A,  as  amended  in  February 
1992,  for  a  specific  flight  training  device  level. 

In  coordination  with  various  entities  within  the  ^  aviation  ^ 
industry,  the  FAA  has  defined  seven  levels  of  flight  training 
devices.  Level  1  through  Level  7.  Level  1  is  currently _ reserved. 
Levels  2  and  3  are  generic  in  that  they  are  representative  of  no 
specific  airplane  cockpit  and  do  not  require  reference  to  a 
specific  airplane.  Levels  4  through  7  represent  a  specific 
cockpit  for  the  airplane  represented.  Within  a  specific 
category,  each  higher  level  of  flight  training  device  is 
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progressively  more  complex.  Because  of  the  increase  in 
complexity  and  more  demanding  standards  when  progressing  from 
Level  2  to  Level  7,  there  is  a  continuum  of  technical 
capabilities  across  those  levels.  For  a  more  detailed 
description  of  the  different  levels  of  flight  training  devices,  a 
table  of  minimum  standards  (from  FAA-AC-120-45A)  for  each  level 
is  presented  in  appendix  A. 

2.3  COMPUTER-BASED  SIMULATION  DEVICE. 

A  computer-based  simulation  device  is  a  microcomputer  that 
utilizes  a  cathode-ray  tube  (CRT)  display,  keyboard,  and 
joysticks  to  simulate  the  operational  aspects  of  the  flight  deck 
environment.  These  simulation  devices  usually  contain  a  rough 
model  of  aircraft  dynamics,  displays  to  represent  cockpit 
instruments,  and  joysticks  to  control  pitch,  roll,  and  thrust. 
Also,  a  keyboard  is  used  to  control  avionics,  aircraft 
configuration  (gear,  flaps,  and  spoilers) ,  and  subsystems 
(electrical,  hydraulic,  etc.).  Ideally,  these  computer-based 
simulation  devices  will  permit  systematic  interaction  between 
subject  and  device,  provide  appropriate  feedback,  and 
automatically  record  a  subject's  performance. 

The  increase  in  utilization  of  computer-based  simulation  devices 
is  a  direct  result  of  the  advancements  in  microcomputer 
technology.  New  technology  has  made  extremely  fast,  high- 
resolution  graphics  available  at  a  low  cost.  A  combination  of 
these  graphics  capabilities  and  the  sophisticated  flight 
simulators  now  being  designed  for  the  microcomputer,  has  provided 
an  inexpensive  means  of  incorporating  some  of  the  activities 
performed  by  a  pilot  into  the  research  setting  (Shappell  and 
Bartosh,  1991)  .  The  possibility  of  obtaining  experimental 
control  and  a  level  of  external  validity  for  associated  cognitive 
tasks  is  a  benefit  of  microcomputer-simulation  experimental 
research.  Computer-based  simulation  devices  range  from  simple 
interactive  software  representations  of  the  out-the-window  flight 
environment,  to  complex  multidisplay  devices  capable  of 
representing  every  component  normally  found  within  an  aircraft 
cockpit.  This  level  of  variability,  in  terms  of  sophistication, 
has  deterred  any  efforts  to  classify  computer-based  simulation 
devices  in  the  same  manner  as  was  introduced  for  airplane 
simulators  and  airplane  flight  training  devices. 

2.4  SUMMARY. 

The  use  of  simulation  for  research  is  concerned  with  other  issues 
in  addition  to  physical  realism,  such  as  the  level  of  realism 
perceived  by  the  pilot  (perceptual  fidelity) ,  and  therefore  can 
not  be  classified  in  accordance  to  the  specifications  outlined 
above.  This  lack  of  classification  for  research  simulation 
devices  has  lead  to  confusion  when  specifying  what  type  of 
simulation  is  necessary  for  a  particular  research  task,  and  is 
the  major  reason  for  performing  this  analysis  of  fidelity 
requirements  for  simulation  research. 
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The  increased  importance  of  the  aforementioned  simulation  devices 
is  due  to  a  variety  of  factors,  including  cost  and  time  savings 
that  can  be  realized,  the  ability  to  reproduce  and  examine 
situations  that  would  be  unsafe  using  actual  equipment,  and  the 
control  and  measurement  of  human-machine  performance . _  Additional 
impetus  has  been  provided  by  the  tremendous  technological  ^ 
advances  in  computer  software  and  hardware  capabilities  being 
incorporated  into  these  devices.  These  robust  capabilities  are 
being  obtained  at  the  expense  of  increased _ developmental , 
operational,  and  maintenance  costs.  This  increased  cost  has 
caused  flight  simulator  users  to  look  closely  at  the  simulator 
requirements  necessary  to  effectively  perform  their  tasks,  and 
then  invest  enough  resources  to  obtain  a  simulation  device  that 
meets  their  specified  needs.  The  remainder  of  this  doc^ent  will 
discuss  simulation  requirements  in  terms  of  fidelity  and  how  to 
determine  what  requirements  are  necessary  for  a  specific  tasK. 


3 ■  SIMULATION  FIDELITY . 

The  concept  of  fidelity  relates  to  the  degree  to  which  the 
characteristics  of  a  flight  simulator  match  those  of  the  real 
airplane.  The  issue  of  simulator  fidelity  has  been  discussed  and 
studied  for  over  30  years,  and  there  is  still  no  agreed  upon  _ 
single  definition.  During  this  time,  the  term  has  been  used  in 
variety  of  ways  and  to  refer  to  many  different  aspects  of 
simulation.  Attempts  to  make  the  term  less  vague  have  caused 
numbers  of  definitions  to  proliferate.  Lane  and  Alluisi  (1992) 
indicated  that  at  least  22  different  definitions  have  to  be  used 
in  the  literature  to  refer  to  different  kinds  of  fidelity, 
representative  sample  of  the  different  kinds  of  fidelity  inclu  es 
such  things  as:  equipment  fidelity,  environmental  fidelity, 
psychological  fidelity,  task  fidelity,  physical  fidelity, 
fuLtional  fidelity,  and  so  on  (Allen,  Buffardi  &  Hays,  1991). 
Each  of  these  kinds  of  fidelity  could  be  appropriate _ for  a 
particular  application,  but  each  are  not  individually  applicable 
to  overall  aircraft  simulation  in  general.  A  common  thread, 
however,  is  that  together  they  imply  at  least  two  ma^or  features 
or  dimensions  along  which  these  simulated  devices  may  differ  from 
actual  equipment . 

These  two  features,  as  defined  by  the  Advisory  Group  ^or  _  _ 

Aerospace  Research  &  Desi  (AGARD)  in  1980,  reppsent  the  division 
of  a  simulator  into  two  classes  depending  on  the  nature  of  the 
cues  they  provide . 

a.  Equipment  cues  provide  a  duplication _ of  the  appearance 
and  feel  of  the  operational  equipment  (the  aircraf t ) ,  i . e . ,  the 
static  and  internal  dynamic  characteristics  such  as  the  size, 
shape,  location,  and  color  of  controls  and  displays,  including 
controller  force  and  displacement  characteristics. 

b.  Environment  cues  provide  a  duplication  of  environment  and 
motion  through  the  environment. 
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Fidelity  is  then  a  function  of  the  degree  to  which  the  equipment 
and  environmental  cues  relate  to  those  of  the  real  airplane.  A 
distinction  between  the  real  cues,  measured  objectively,  and  the 
cues  the  pilot  subjectively  experiences,  provides  the  following 
definitions  for  two  types  of  fidelity  (AGARD,  1980)  : 

a.  Objective  Fidelity  provides  an  engineering  standard  and 
is  the  degree  to  which  a  simulator  would  be  observed  to  reproduce 
its  real-life  counterpart  aircraft,  in  flight,  if  its  form, 
substance,  and  behavior  were  sensed  and  recorded  by  a 
nonphysiological  instrumentation  system  onboard  the  simulator. 

By  including  both  equipment  and  environmental  cues,  this 
definition  can  encompass  all  pertinent  dynamic  cue  timing  and 
synchronization  aspects  of  simulator  fidelity. 

b.  Perceptual  Fidelity  provides  a  psychological/ 
physiological  standard  and  is  the  degree  to  which  the  flight  crew 
subjectively  perceives  the  simulator  to  reproduce  its  real-life 
counterpart  aircraft,  in  flight,  in  the  operational  task 
situation.  The  requirement  that  the  operational  equipment  be 
considered  in  the  context  of  the  task  situation  ensures  that  not 
only  cue  timing  and  synchronization,  but  also  cue  priority 
effects,  are  taken  into  account. 

Among  the  various  categories  of  simulators,  there  is  a  tradeoff 
between  equipment  and  environmental  cue  fidelity  required.  The 
physical  correspondence  between  simulator  and  airplane  in  terms 
of  cockpit  layout,  flight  instruments,  controls,  etc.,  for 
training  simulators  should  result  in  high  equipment  cue  fidelity 
to  advocate  a  high  degree  of  transfer  of  training  to  the 
operational  environment.  On  the  other  hand,  research  and 
development  simulators  should  place  more  emphasis  on  high 
environmental  cue  fidelity.  Environmental  cues,  by  definition, 
provide  duplication  of  the  operational  aircraft  environment  and 
motion,  and  thus  will  result  in  a  higher  degree  of  realism  being 
experienced  by  the  subjects.  This  perceived  realism  will  result 
in  subject  performance  more  closely  matching  that  which  would 
occur  in  the  real  world.  Figure  1  shows  the  requirements  for 
equipment  and  environmental  cue  fidelity  (AGARD,  1980) . 

Within  the  types  of  simulators  indicated  in  figure  1,  the  levels 
of  fidelity  can  vary  greatly.  For  example,  having  cockpit  crew 
coordination  as  the  simulation  task,  a  work  station  can  be 
defined  as  a  relatively  low  fidelity  research  simulator.  Yet, 
another  research  simulator,  such  as  those  found  at  the  Crew 
Station  Research  and  Development  Facility  (CRSDF)  located  at 
NASA-Ames,  certainly  have  a  higher  level  of  fidelity  for 
evaluating  the  same  task.  Thus,  for  a  specified  research  task, 
the  user  must  be  able  to  determine  fidelity  requirements.  Failure 
to  properly  determine  these  requirements  can  result  in  (1) 
unsatisfactory  results  due  to  lack  of  fidelity,  or  (2) 
satisfactory  results  but  at  premium  cost  (suboptimization) 
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(Prasad  et  al . ,  1991).  The  problem  now  is  how  to  define  specific 
fidelity  requirements  for  a  simulation  task. 


FIGURE  1.  REQUIREMENT  FOR  EQUIPMENT  AND  ENVIRONMENTAL 
CUE  FIDELITY  (FROM  AGARD,  1980) 


3 . 1  FIDELITY  DRIVERS . 

The  previous  section  discussed  the  concept  of  simulator  fidelity 
and  introduced  its  two  most  important  components;  objective  and 
perceptual  fidelity.  Also,  emphasis  was  put  on  the  need  to 
specify  fidelity  requirements.  These  rec^irements  can  be 
obtained  by  basing  decisions  on  the  configuration  of  the 
simulation  system  and  the  systematic  rational  examination  of  how 
the  specific  simulation  is  to  be  used  (Lane  &  Alluisi,  1992) .  In 
that  context,  generalities  about  the  pros  and  cons  of  high  an 
low  fidelity  are  not  very  helpful.  Therefore,  Lane  and  Alluisi 
(1992),  developed  four  key  dimensions  or  fidelity  drivers  to  be 
used  for  the  examination  of  simulation  requirements.  The  four 
key  fidelity  drivers  are  identified  and  discussed  below. 

a.  Mission (s)  or  Mission  Segment  to  be  Simulated  -  For 
realistic  practice  or  evaluations  to  occur,  the  system  must  be 
uLd  to  perform  some  mission.  A  researcher  may  wish  to  simulate 
all  mission  phases  or  (frequently)  only  selected  segments 
mission.  The  segments  that  are  included  in  the  simulation  wi 
dictate  the  specific  tasks  to  be  performed  by  the  operator,  the 
system  components  involved  in  performing  these  tasks,  and  thus 
the  simulation  components  on  which  fidelity  should  be  focused. 

b.  Obiectives  of  the  Simulation  -  A  simulation  is  intended 
(1)  to  provide  practice  on  specific  skills,  (2)  to  rein  orce 
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acquisition  and  use  of  job-relevant  knowledge,  or  (3)  to  evaluate 
a  system  or  a  new  concept.  These  potential  objectives  can  be 
described  in  terms  of  broad  classes  of  basic  operator  activities 
that  the  simulation  does  or  does  not  need  to  support.  The 
fidelity  required  to  meet  specific  objectives  would  be  based  on 
the  extent  to  which  each  of  the  identified  activities  that  occur 
within  a  mission  segment  must  be  supported  by  the  simulation,  and 
in  what  detail. 

c.  Fidelity  Dimensions  -  The  dimensions  on  which  fidelity 
can  be  examined  and  evaluated  can  be  grouped  into  three  general 
classes:  there  are  dimensions  that  show  and  describe  the 
attributes  of  (1)  the  simulator,  (2)  the  operator,  and  (3)  the 
processes  or  events  external  to  the  simulator  itself.  The  first 
class  of  attributes  describes  the  simulated  system  itself,  and 
the  fidelity  concerns  are  those  that  address  the  operator's 
equipment  and  its  immediate  environment --the  look  and  feel  that 
are  a  result  of  the  physical,  sensory,  and  perceptual  variables 
employed.  The  second  class  of  attributes  are  drivers  that 
determine  the  specific  tasks  to  be  performed  by  the  operator  and 
the  task  loading  under  which  the  operator  will  work.  These 
drivers  are  external  to  the  system- -what  the  operator  does  in  the 
simulation  is  determined  by  how  the  simulated  system  is  used. 

The  third  class  of  fidelity  attributes  are  concerned  with  the 
external  processes  that  generally  arise  from  the  dynamics  of 
system  participation  in  a  variety  of  interactive  events .  Each  of 
these  attributes  and  characteristics  defines  a  dimension  of 
fidelity,  and  each  requires  decisions  about  the  level  of  fidelity 
at  which  that  attribute  will  be  represented  for  a  component  of 
the  simulation  within  the  context  defined  by  the  missions  and 
objectives  of  the  simulation. 

d.  Simulation  Components  -  The  execution  of  simulation 
requires  the  presence  of  numerous  individual  components .  These 
components  are  subdivided  into  local  and  global  sets .  Local 
components  are  part  of  the  simulator  and  its  immediate 
environment;  they  have  roughly  the  same  characteristics  as  the 
three  classes  of  fidelity  dimensions  defined  above.  Global 
components  are  defined  by  the  external  processes  and  environments 
with  which  the  local  components  interact.  The  importance  of 
breaking  down  a  simulation  into  its  building  blocks  is  that  it  is 
about  these  individual  components  that  fidelity  requirement 
decisions  must  be  made. 

These  four  drivers  of  fidelity  constitute  four  dimensions  on 
which  any  given  simulation  can  be  analyzed  (Lane  &  Alluisi, 

1992)  .  The  ultimate  objective  is  to  determine,  for  each 
component,  on  each  fidelity  dimension,  the  degree  of  fidelity 
required  to  support  the  intended  uses  of  the  simulation. 

3 . 2  FIDELITY  OF  SIMULATION  DEVICE  SUBSYSTEMS . 

The  basic  type  of  characteristics  that  drive  simulator  fidelity 
requirements  have  been  outlined.  However,  definitions  still  do 
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not  exist  for  the  specific  physical  components  of  a  simulator  to 
analyze  when  determining  fidelity  requirements.  Prasad  et  al . , 
(1991)  performed  a  survey  of  simulation  devices  and  existing 
technologies  and  determined  that  there  are  generally  ten 
subsystems,  shown  in  table  1.,  which  adequately  describe  a  given 
simulator.  Table  2  describes  fidelity  characteristics. 

TABLE  1.  SIMULATOR  SUBSYSTEMS 


(1)  Cockpit 

( 2 )  Audio 

(3)  Motion 

(4)  Control  System 

( 5 )  Math  Model _ 


(6)  Environment 

(7)  Ground  Handling 

(8)  Mission  Equipment 

(9)  System  Latency 

(10)  Visual _ 


TABLE  2.  LEVELS  OF  FIDELITY  CHARACTERISTICS  FOR  SIMULATOR 
SUBSYSTEMS 


(1) 

Cockpit/Crew  Station 

-  none 

-  simulated/generic  type  instruments 

-  partially  simulated  cockpit 
full  up  crew  station 

(2) 

Audio 

-  none 

-  significant  cockpit  sounds 

-  incidental  sounds  (precip. ,  etc. . . ) 

-  realistic  - - 

(3) 

Motion 

-  none 

-  2D0F  (pitch  and  roll) 

-  3DOF  (pitch,  roll,  and  yaw) 

-  6  DOF 

(4) 

Control  System 

-  no  force  feel 

-  constant  force  ( spring /damper ) 

-  partial  duplication  of  actual  force 
complete  duplication 

(5) 

Mathematical  Model 

-  none 

-  3  DOF 

-  6  DOF 

6  DOF  with  rotor 

(6) 

Environment 

-  clean  air 

-  discrete  gusts 

-  first  order  filtered  turbulence 

-  rotationallv  sampled  turbulence 

(7) 

Ground  Handling 

~  no  gear 

-  rigid  gear 

-  simplified  gear  model 

_ -  comprehensive _ _ _ _ _ _ _ 

(8) 

Mission  Equipment 

-  none 

-  communication  only 

-  communication/navigation  only 

_ ~  complete _ _ _ _ _ _ _ 

(9) 

System  Latency 

-  non  real  time  (off  line) 

-  significant  delay 

-  minimal  delay 

_  -  real  time _ _ _ _ _ _ _ — 

(10)  visual 

of  view  dynamic  ranqe  detail 

workstation  day  low 

75°horiz/35°vert  dusk  medium 

90°horiz/40°vert  haze/fog  high 

wider  night  very  high 
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For  each  subsystem,  it  is  possible  to  associate  a  level  of 
objective  fidelity  with  the  degree  of  equipment /software 
sophistication.  For  example,  a  simulator  with  day,  night,  and 
dusk  visual  capability  with  a  wide  field  of  view  can  be 
associated  with  high  fidelity,  while  a  simulator  with  just  night 
visual  capability  can  be  associated  with  low  fidelity.  This  _ 
association  between  fidelity  and  the  subsystems  defines  fidelity 
characteristics;  this  can  provide  the  necessary  information  for 
determining  actual  simulator  characteristics  needed  to  obtain  a 
certain  level  of  fidelity.  Listed  below  are  the  fidelity 
characteristics  (rank  order;  none  to  high)  of  the  simulator 
subsystems  that  span  the  spectrum  of  fidelity  (Prasad  et  al . , 

1991)  . 

Because  of  the  extensive  use  of  flight  simulators  as  both 
training  and  engineering  evaluation  devices,  considerable  effort 
has  been  devoted  to  determining  the  differences  that  exist 
between  the  simulation  and  in-flight  environment.  The  information 
provided  in  the  table  gives  a  description  of  simulator  fidelity 
and  how  the  simulator  components  vary  in  levels  of  fidelity. 

4.  RESEARCH  SIMULATOR  FIDELITY  REQUIREMENTS- 

Up  to  this  point,  this  document  has  introduced  the  various  types 
of  aviation  simulation  research  devices  and  described  the  concept 
of  simulator  fidelity.  This  information  can  be  used  to _ determine 
how  to  manipulate  simulator  fidelity  and  how  various  simulator 
characteristics  affect  fidelity,  but  no  recommendations  for 
simulator  fidelity  requirements  necessary  to  achieve  a  desired 
^g3g3]fch  goal  have  been  introduced.  This  section  will  examine 
general  fidelity  requirements  for  experimental  simulation 
research. 

The  essential  feature  of  simulator  experimental  investigations  is 
to  introduce  the  pilots  into  a  closed  loop  control  situation,  so 
that  account  is  taken  of  their  capabilities  and  limitations 
regarding  the  performance  measure  being  evaluated. ^  The 
expectation  is  that  within  the  bounds  of  the  experimental 
conditions,  behavior  in  the  simulator  will  match  their  behavior 
in  the  flight  situation.  Hence,  the  primary  goal  for  a  flight 
simulation  researcher  is  to  produce  experimental  conditions  that 
elicit  behavior  that  would  occur  under  similar  circumstances  in 
the  real  world.  The  ultimate  consideration  is  performance  in  the 
real  world  (Orlady,  Hennessy,  Obermayer,  Vreuls  &  Murphy,  1988) . 
This  experimental  construct  for  human  factors  research  was 
alluded  to  by  David  Meister  (1985)  when  he  wrote: 

...the  purpose  of  ergonomic /human  factors  is  to  describe, 
analyze,  measure,  predict,  and  control  the  real  world  of  systems 
functioning  operationally  (i.e.,  not  under  experimental  control). 

This  statement  results  in  further  direction  for  human  factors 
researchers,  again  in  the  words  of  Meister  (1985) . 
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.in  consequence,  the  ideal  environment  in  which  to 
gather  data  is  the  operational  environment.  It  may 
for  various  reasons  to  measure  in  some  environment  other  than  t 
real  world,  such  as  a  laboratory  or  a  simulator  but  such 
cases  the  conclusions  derived  from  the  data  must  be  verified 
the  operational  environment. 

Meister's  comments  must  always  be  considered  during  the 
determination  of  fidelity  requirements  for  simulated  research. 
Regardless  of  the  level  of  fidelity  and  how  accurately  it  was 
determined  to  meet  the  goals  of  the  research,  no  simulator 
evaluation  can  completely  duplicate  the  experience  in  the  real 
world.  Therefore,  any  results  should  be  verified  operationally 
j)0fore  definite  conclusions  can  be  made. 

4 . 1  DETERMINING}  FIDELITY  REQUIREMENTS  . 

Fidelity,  as  defined  earlier,  is  a  multifaceted  concept  and  is 
dependent  on  a  variety  of  different  simulator  components.  Also, 
fidelity  is  presently  a  metric-free  construct  with  no  agree  upo 
measurement  scale  on  which  the  fidelity  of  a  specific  simulation 
can  be  located  and  assigned  a  numerical  value  (Lane  &  Allu  , 
1992) .  Furthermore,  the  requirements  for  fidelity  change  fro 
one  research  effort  to  another,  depending  on  what  type  of 
experimental  study  (part-task,  full-mission,  etc.)  is  being 
implemented  or  the  types  of  measures  being  . 

aspects  of  simulator  fidelity  support  the  general  claim  within 
the  simulation  community  that  fidelity  requirements  cannot  be 
determined  in  general  for  various  types  of  simulations  and  th 
they  are  dependent  on  the  specific  objective  the  simulation  is 
intended  to  accomplish.  If  decision  making  skills  and  tactics 
are  being  evaluated,  then  high  fidelity  in  vehicle  handling 
characteristics  are  not  critical.  On  the  other  hand,  if  ^ 

interest  is  in  the  examination  of  manual-control  skills,  tne 
answer  is  obviously  different. 

When  planning  human  factors  research,  the  required 
characteristics  and  features  of  the  research  vehicle  are 
prominent  issues  of  consideration  in  the  goal  of  choosing  the 
right  simulator  for  the  planned  research  investigation Orlady  et 
al  (1988)  specified  two  principal  factors  in  determining  e 
choice  of  research  vehicle  (1)  the  type  of  research _ required  by 
the  problem,  and  (2)  knowledge  of  the  factors  that  influence  the 
beL^^jirproce^Ls  of  Interest.  The  type  of  research  dictates 
the  level  of  representation  that  the  research  vehicle  mu 
Knowledge  of  the  factors  which  influence  the  behavioral  processes 
determine  how  comprehensively  the  research _ vehicle  must  represen 
an  operational  system  and  its  associated 

Together,  these  features  are  commonly  thought  of  as  the  fidelity 
of  the  research  vehicle. 

Practical,  as  well  as  scientific,  considerations  help  determine 
the  characteristics  of  a  simulator  used  for  research.  In 
general,  high  fidelity  representations  of  real-world  situations 
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incur  costs  that  are  proportional  to  the  comprehensiveness  and 
complexity  of  the  research  project.  Hence,  a  conflict  arises 
between  the  real-world  complexity  of  the  operational  tasks  and 
the  need  for  economy  of  the  research  effort.  A  second  conflict 
arises  between  the  need  to  elicit  behavior  that  is  ecjuivalent  to 
the  real  world,  and  the  need  to  have  experimental  control  to 
minimize  variability  necessary  to  make  reliable  and  statistically 
significant  conclusions.  The  goal  of  a  simulation  experiment 
design  methodology  is  to  determine  a  reasonable  balance  between 
these  conflicting  experimental  demands. 

Advancements  in  simulation  technology  are  enabling  nearly 
realistic  environments  to  be  utilized  in  experimental  research. 
These  accomplishments  are  providing  continuing  support  towards 
the  historical  goal  of  designing  a  simulator  to  be  equivalent  to 
an  actual  aircraft.  The  view  expressed  by  many  in  the  simulation 
community  is  that  the  usefulness  of  a  simulator  should  only  be 
equated  to  its  degree  of  realism.  For  research,  the  basic 
assumption  is  that  the  more  faithful  the  simulation  of  real-world 
stimuli,  the  interfaces  between  individual  crew  members,  the 
systems  they  control,  and  the  systems  that  influence  and  regulate 
their  behavior,  the  more  likely  it  is  that  the  behavior  achieved 
in  the  experiment  will  be  the  behavior  that  would  be  produced 
under  similar  circumstances  in  the  operational  environment 
(Orlady  et  al.,  1988). 

A  high  level  of  fidelity  has  another  advantage  when  used  in 
research  simulations.  It  increases  the  face  validity  of  the 
experiment  in  the  eyes  of  both  the  participants  in  the  experiment 
and  in  the  potential  users  of  the  research  (Orlady  et  al . ,  1988) . 
Face  validity,  or  an  apparent  high  level  of  real-world 
representation,  in  a  simulation  will  allow  the  users  in  the 
aviation  community  to  have  more  confidence  in  the  experimental 
results . 

When  considering  levels  of  fidelity  for  research  it  is  often 
assumed  that  high  fidelity  is  never  a  disadvantage,  as  is 
apparent  in  the  philosophy  stated  by  O'Hare  and  Roscoe  (1990)  as: 
"If  you're  not  sure  what's  important,  play  safe  and  buy  the  most 
fidelity  possible;  surely  something  will  work."  There  are, 
however,  research  as  well  as  practical  reasons  not  always  to ^ 
strive  for  maximum  fidelity.  In  general,  high  fidelity  implies  a 
comprehensive  representation  of  the  real  world.  Furthermore,  it 
provides  an  opportunity  for  unknown,  extraneous  factors  to 
influence  behavior,  and  gives  subjects  an  opportunity  to  choose 
behavioral  alternatives  that  may  be  beyond  the  research  scope  of 
interest.  These  potential  consequences  of  high  fidelity 
simulation  contradict  the  two  basic  principles  of  behavioral 
research,  as  stated  by  Orlady  et  al . ,  1988,  which  are  (1) 
maintain  control  of  the  research  situation,  and  (2)  to  account 
for  the  factors  which  influenced  the  observed  behavior. 
Unnecessarily  high  levels  of  fidelity  can  complicate  the  study 
and  hinder  the  researcher's  ability  to  investigate  a  specific 
research  issue.  The  effects  of  high  fidelity  can  often  show  up 
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as  variability  in  the  data  and  reduce  the  sensitivity  of  the 
performance  measures  as  well  as  the  reliability  of  their  values 


However,  it  over  simplifies  the  issue  to  say  that  a  simulation 
should  have  all,  but  no  more  than,  those  characteristics  that 
directly  affect  the  behavior  being  examined.  This  is  an  ideal 
goal,  but  one  that  rarely  can  be  achieved.  Researchers  too  often 
include  irrelevant  capabilities  into  a  simulation  in  an  effort  to 
increase  fidelity.  Attention  should  be  given  to  maintaining 
control  of  the  experimental  situation,  and  the  variability  in 
subject  behavior  and  performance  that  might  result  because  of  the 
presence  of  extraneous  factors. 


In  conclusion,  the  concept  of  simulation  fidelity,  although  in 
widespread  use,  is  difficult  to  quantify,  especially  for  research 
simulators.  Additionally,  the  amount  of  research _ available  that 
investigates  fidelity  requirements  for  research  simulators  is  not 
abundant,  and  no  real  guidelines  have  ever  been  agreed  upon  by 
simulator  researchers.  Given  this  lack  of  significant  researc 
in  the  area  and  no  generally  accepted  consensus,  it  is  difficult 
to  determine  specific  fidelity  requirements  for  research 
applications.  To  reiterate,  simulation  fidelity  requirements  are 
dependent  specifically  on  what  the  simulator  is  to  be  used  for. 


S ■  HUMAN  FACTORS  RESEARCH  FIDELITY  CONCERNS . 

Simulation  researchers  are  faced  with  the  many  problems, 
considerations,  and  conflicting  issues  relating  to  the 
determination  of  fidelity  requirements  for  a  simulation  research 
effort.  As  mentioned  throughout  this  document,  fidelity  is  a 
multivariant  construct  with  no  consensus  among  researchers  of  a 
single  index  of  measurement  or  definition.  Furthermore,  the 
associated  benefits  of  using  both  high  and  low  levels  of  fidelity 
for  aviation  experimental  research  has  been  shown.  Given  the 
degree  of  differences  and  difficulties  relating  to  simulator 
fidelity,  specific  guidelines  for  various  types  of  experimental 
research  are  not  presently  available.  However,  general 
conclusions  can  be  made  regarding  how  varying  levels  of  fidelity 
can  effect  the  outcome  of  different  types  of  experiments. 

This  section  will  define  the  major  types  of  experimental  research 
studies  performed  in  simulators,  and  introduce  the  fideli-y 
concerns  associated  with  each.  Also,  the  types  of  performance 
measures  that  can  be  reliably  investigated  in  research 
simulations  are  discussed.  Finally,  recommendations  based  on  the 
information  provided  herein  are  given  for  specific  types  ot 
simulation  devices  that  can  be  used  for  various  research 
objectives . 

Before  embarking  on  this  introduction  and  discussion  of 
experimental  study  types  and  their  respective  fidelity  concerns, 
a  general  assumption  of  fidelity  for  research  simulators  must  be 
presented.  Research  simulators  as  a  whole  demand  high  levels  of 
fidelity  due  to  their  overall  objective  of  observing  behavior. 
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Research  needs  are  more  rigorous  and  fidelity  requirements  are 
more  strict  so  that  researchers  can  observe  behavior  as  close  to 
the  operational  environment  as  possible.  Deviations  from  the 
behavior  being  studied  provide  variance  that  can  confound 
analysis .  This  high  level  of  fidelity  assumption  is  not 
encompassing  for  research  simulators.  There  are,  as  mentioned 
earlier,  several  instances  where  a  high  level  of  fidelity  can  be 
a  disadvantage,  and  these  research  issues  will  be  discussed 
later.  But,  the  majority  of  information  regarding  fidelity 
requirements  for  experimental  research  studies  addresses  issues 
pertaining  to  high  fidelity  research  devices. 

5.1  SIMULATION  RESEARCH  STUDY  TYPES. 

Experimental  research  studies  performed  using  a  simulation  device 
generally  fall  into  two  categories,  full-mission  and  part- task. 
Full— mission  studies  examine  behavior  in  the  full  context  of  the 
aviation  environment,  while  part- task  studies  concentrate  on  the 
behavior  relating  to  a  specific  task  or  function.  Within  these 
two  categories  there  are  distinct  study  types  that  are 
distinguished  by  the  complexity  of  their  obj ectives  or  the  type 
of  simulation  device  used  for  investigation. 

Full-Mission  Studies 

The  basic  idea  of  full-mission  relates  to  performing  a  research 
study  with  the  most  realistic  simulation  possible.  Full-mission 
simulation,  as  stated  by  Orlady  et  al . ,  (1988),  includes  all  of 

the  stimuli  presented  to  the  flight  crew.  It  includes  the 
aircraft  cockpit,  visual  and  motion  cues,  aircraft  flight 
dynamics,  all  of  the  aircraft  subsystems,  the  flight  environment 
(including  air  traffic  control  (ATC) ,  weather,  and  other  air  or 
ground  vehicular  traffic)  ,  the  cabin  crew,  and  all  ancillary 
flight  services  (such  as  dispatch,  ramp  passenger  services,  and 
maintenance) . 

A  full-mission  simulation  study  can  involve  numerous  simulators 
in  the  same  study  environment  connected  together  over  a  network, 
known  as  an  Interactive  Mission  Scenario  (Prasad  et  al . ,  1991). 
Also,  a  full-mission  simulation  that  examines  behavior  over  an_ 
entire  flight  from  preflight  checklist  to  parking  at  the  gate  is 
considered  an  end-to-end  simulation  experimental  study. 

Given  the  above  definition  of  full-mission  simulation  and  the 
types  of  experimental  studies  mentioned  that  utilize  this 
technique,  it  can  be  accurately  stated  that  only  airplane 
simulators,  as  defined  in  section  2. ,  can  be  used  for  full- 
mission  simulation  applications.  The  airplane  simulators  are  the 
only  simulation  devices  that  specify  visual  and  motion 
characteristics  by  definition,  and  these  capabilities  are ^ 
definitely  necessary  when  conducting  full-mission  simulation 

oh .  The  overall  fidelity  associated  with  the  full— mission 
simulation  is  related  to  what  level  (A,  B,  C,  or  D)  of  airplane 
simulators  is  used  in  the  study.  A  main  fidelity  issue  of 
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concern  in  full-mission  simulation  is  the  pilot  subjects  desire 
for  scenario  fidelity.  Pilots  generally  do  not  accept  deviations 
f?L  operational  practice  unless  it  is  specified  at  the  beginning 
of  the  simulation.  Pilots  experiencing  negative  user  acceptance 
cues  are  more  likely  not  to  elicit  the  same  behavior  as  they 
might  in  the  real  world. 

The  full-mission  simulation  type  of  devices  can  be  used  to 
investigate  cognitive  tasks  in  the  context  of  the  multitask, 
complex  operation  of  flying  an  airplane.  Also,  much  of  tne 
research  performed  using  full-mission  simulation  focuses  on 
cockpit  instrumentation,  crew  procedures,  and  workload  _ 
measurement.  Topics  of  investigation  include  how  decisions  mad 
in  the  cockpit  are  affected  by  environmental  and  hardware 
difficulties  as  well  as  by  the  availability  of  information  from 
ATC  and  other  aircraft,  how  errors  are  made,  and  the  effects  o 
automation,  fatigue,  and  advanced  instrumentation  of  human 
performance  (Jones,  Hennessy,  and  Deutsch,  1985). 

Orlady  et  al . ,  (1988)  indicated  four  principal  reasons  for  doing 

full -miss ion  simulation : 

1.  To  resolve  a  collection  of  related  problems  -  if  there  is 
a  series  of  part-task  evaluations  planned,  it  may  be 
economical  to  group  them  together  in  a  comprehensive  study. 

2.  When  the  focus  of  interest  is  on  long  duration  or 

infrequent  events  and  effects  -  behavior  under  ^  . 

responses  to  rare  emergencies  as  a  function  of  time  are  obvious 

examples . 

3  Subtle  interactions  may  influence  the  behavior  of 

interest  -  results  of  a  crew  coordination,  . 

or  resource  management  study  are  likely  to  be  adversely  affect 
if  the  simulation  is  not  physically  comprehensive  and  realistic, 
or  if  scenario  is  too  short. 

4  To  evaluate  performance  of  people  and/or  equipment  that 
occurs  during  a  series  of  transitions  from  one  flight  phase  of 
Operation  to  another . 

These  capabilities  of  full-mission  simulation  show  the  importance 
this  technique  has  within  human  factors  simulation  researc  . 
However,  there  may  be  types  of  studies  which  can  be  accomplished 
more  efficiently  using  other  research  methods  with  lower  fidelity 
Simulation  devices.  ?he  problem  is  that  researchers  are  unsure 
about  the  level  of  fidelity  necessary,  and  too  often  elect  the 
safer  route  of  full-mission  simulation  with  high  fidelity. 

Part-Task  Studies 

The  concept  of  a  part-task  study  is  to  investigate  a  performance 
S  responie  to  a  specific  manipulated  task  or  functron. 
Part-task  simulation  is  characterized  by  a  functional 
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representation  of  a  specific  subsystem.  These  types  of 
simulation  devices  isolate  a  single  critical  function  for 
evaluation  in  terms  of  pilot  behavior.  For  example,  a  part-task 
study  may  only  evaluate  specific  instruments,  displays, 
operational  procedures,  or  controls  found  in  the  cockpit.  The 
benefit  of  part-task  simulation  is  derived  from  the  view  that  the 
evaluation  of  smaller  component  tasks  is  more  acceptable  for  _ 
experimental  testing  and  statistical  analysis  and  more  objective 
information  regarding  performance  can  be  obtained.  However,  what 
part-task  simulation  may  gain  in  experimental  control,  it  lacks 
in  external  validity,  i.e.,  accurate  representation  of  the  real 
world. 

Part-task  simulation  is  appropriate  for  research  when  there  is  no 
reason  to  suspect  behavior  wi±l  be  influenced  by  seccpndary 
contextual  circumstances  (Orlady  et  al.,  1988) .  Basically, 
studies  looking  at  human  performance  on  a  specific  individual 
task  (reaction  time,  accuracy,  etc.),  or  functional  problems 
inherent  to  a  task  or  condition  can  be  evaluated  using  part-task 
simulation.  Furthermore,  as  stated  earlier,  extraneous  factors 
may  add  unwanted  variance  to  the  particular  problem  being 
researched. 

Devices  used  for  part-task  studies  range  from  cardboard  cutouts 
i-0px‘0S0nt ing  cockpit  displays,  to  the  most  sophisticated  flight 
training  device.  Also,  the  technology  advancements  _ 

microcomputer  graphics  is  resulting  in  an  increased  utilization 
for  part-task  research.  Computer-based  simulators  can  present 
real-life  replicas  of  cockpit  displays  and  allow  subjects  to _ 
interact  with  the  cockpit  as  they  would  during  normal  operations . 
The  fidelity  characteristics  of  part-task  simulations  are  related 
to  the  graphics  capability  of  the  microcomputer  and  the  level  of 
flight  training  device  1-7,  as  defined  earlier,  being  used  in  the 
research.  Some  part- task  research  requires  only  minimal  real- _ 
world  fidelity  and  limited,  but  highly  specialized  expertise  with 
if0g'a]fds  to  subject  capabilities.  Other  kinds  of  part  — task 
research  might  demand  only  a  simple  scenario  to  investigate  a 
fundamental  decision  process,  but  little  in  the  way  of  a  real 
cockpit  or  visual  system. 

5.2  SIMULATION  RESEARCH  FIDELITY  RECOMMENDATIONS. 

As  mentioned  throughout  this  document,  there  are  no  specific 
guidelines  for  determining  simulator  fidelity  requirements _ for 
human  factors  research.  Fidelity  is  dependent  on  the  individual 
goal  and  area  of  investigation  for  a  research  study.  However, 
general  capabilities  and  limitations  of  various  simulation 
(5,0vices  and  their  associated  levels  of  fidelity  can  be 
summarized.  This  section  combines  the  pertinent  information  found 
in  this  analysis  regarding  research  simulation  device 
specifications,  fidelity  requirements  for  research,  and  the 
characteristics  of  the  different  types  of  experimental  research 
simulation  techniques.  This  information  is  used  to  summarize 
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some  general  recommendations  concerning  the  levels  of  fidelity 
]3^0(;;j0ssary  to  properly  conduct  human  factors  simulation  research. 

Full-Mission  Studies 

When  investigating  proper  fidelity  levels  for  full-mission 
simulation  studies  a  researcher  would  generally  desire  the 
highest  level  of  fidelity  available.  Full-mission  simulation,  by 
definition,  requires  all  stimuli  present  in  the  operational 
environment  and  it  has  been  shown  that  the  more  the  simulator 
characteristics  match  those  of  the  real  world,  the  more  subject 
behavior  matches  actual  behavior  that  would  occur  operationally. 

Full-mission  simulation  can  be  used  to  investigate  cognitive 
tasks  in  the  context  of  the  multitask,  complex  operation  of 
flying  an  airplane.  Specifically,  full-mission  simulation  human 
factors  research  focuses  on  cockpit  instrumentation,  crew 
procedures,  and  workload  measurement.  Furthermore,  topics 
concerning  how  decisions  made  in  the  cockpit  are  affected  by 
environmental  and  hardware  difficulties,  how  errors  are  made,  and 
the  effects  of  automation,  fatigue,  and  advanced  instrumentation 
on  human  performance  can  be  investigated  using  full-mission 
simulation.  Additional  uses  of  and  reasons  for  utilizing  full- 
mission  simulation  were  mentioned  earlier. 

The  restriction  that  full-mission  simulation  only  be  performed 
using  flight  simulators  narrows  the  issue  of  determining  an 
adequate  simulation  device,  but  there  still  is  the  varying  level 
of  fidelity  among  the  four  levels  of  airplane  simulators.  Full- 
mission  simulation  can  be  performed  in  any  level  simulator  A 
through  D,  given  their  respective  characteristics  and 
capabilities.  However,  as  the  level  of  simulator  fidelity 
increases,  there  is  also  an  increase  in  the  capital  investment 
required  to  perform  the  research.  Therefore,  a  tradeoff  exists 
between  the  desire  to  perform  research  as  economically  as 
possible,  and  the  need  to  obtain  valid  results  that  represent 
behavior  in  the  operational  environment.  In  any ^ event,  the ^ 
Higher  the  level  of  simulator  fidelity  utilized  in  a  full— mission 
simulation  research  study,  the  more  accurately  a  researcher  can 
make  generalizations  about  behavior  in  the  real  world. 

This  rule  of  thumb  for  full— mission  simulation  research  studies 
is  a  general  recommendation  associated  with  the  utilization  of 
different  levels  of  simulators.  Specific  guidelines  for  minimum 
fidelity  requirements  that  may  vary  from  one  level  cpf  simulator 
to  another  depending  on  the  objective  of  the  full-mission 
simulation  research  cannot  be  accurately  determined,  as  alluded 
to  throughout  this  document.  The  only  guideline  available  for 
full-mission  simulation  is  that  the  research  results  become  more 
generalizable  as  the  level  of  fidelity  increases. 
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Part-Task  Studies 


Determining  of  fidelity  requirements  for  part-task  simulations  is 
more  difficult  and  therefore  more  complicated  than  the  general 
rule  presented  for  full-mission  simulations.  In  general ,  part- 
task  studies  investigate  a  dependent  performance  measure  in 
response  to  a  specific  manipulated  task.  Consequently,  part- task 
simulations  are  appropriate  to  measure  performance  on  a  specific 
task  when  no  other  conditions  are  believed  to  influence  behavior. 
For  example,  if  a  researcher  is  interested  in  measuring  hi^an 
performance  on  an  individual  dependent  task  such  as  reaction  time 
or  accuracy,  a  part-task  study  would  allow  the  behavior  to  be 
measured  individually  with  no  other  conflicting  factors 
influencing  behavior. 

Part-task  simulations  can  utilize  a  wide  variety  of  simulation 
devices  such  as:  microcomputer  simulation  devices,  low— fidelity 
desktop  training  devices,  and  high  fidelity  airplane  flight 
training  devices.  This  wide  range  of  fidelity  devices 
complicates  the  issue  of  determining  specific  requirements  for 
part-task  simulation  fidelity.  The  utilization  of  too  much 
fidelity  can  result  in  unwanted  variance  associated  to  the 
behavior  being  examined.  On  the  other  hand,  if  the  simulation 
does  not  represent  the  context  in  which  the  specific  task  is  to 
be  investigated,  due  to  a  low  level  of  fidelity,  the  behavior 
examined  may  not  be  exactly  that  for  which  the  research  was 
intended.  Hence,  fidelity  requirements  for  part— task  simulations 
cannot  be  determined  in  general.  The  requirements  must  be^ 
determined  on  a  case— by— case  basis  depending  on  the  objectives  of 
the  research. 

If  a  researcher  is  interested  in  a  baseline  human  performance^ 
measure  such  as  rate  of  errors  or  reaction  time,  or  in  comparing 
display  formats  for  user-acceptance,  rudimentary  flight  training 
devices  and  computer-based  simulation  devices  could  be  used  to 
examine  behavior.  These  types  of  part-task  studies  do  not 
require  the  behavior  to  be  measured  in  the  full  airplane  context . 
Conversely,  a  high  fidelity  flight  training  device  would  be 
necessary  to  evaluate  operational  procedures  such  as  preflight  or 
approach  checklists  for  a  specific  airplane.  To  perform  these 
pr'ocedures ,  subject  pilots  would  need  a  complete  replication  of 
the  specified  aircraft's  cockpit.  These  capabilities  are  only 
found  in  the  most  sophisticated  flight  training  devices,  levels 
(4-7)  as  defined  earlier. 

Other  examples,  such  as  the  one  just  mentioned,  could  be 
presented  to  indicate  specific  types  of  research  that  could  be 
adequately  performed  in  the  various  part— task  simulation  devices . 
However,  no  general  guidelines  are  applicable  for  the  full  range 
of  part-task  simulations.  Each  individual  study  has  its  own 
characteristics  and  objectives  and  therefore  its  own  fidelity 
requirements  based  on  the  context  that  a  specific  behavior  must 
be  measured  within. 
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The  aforementioned  recommendations  for  part- task  simulation 
fidelity  requirements  were  not  very  specific.  As  the  main 
problems  associated  with  simulation  fidelity  requirements 
mentioned  in  this  document  were  brought  to  the  forefront  for 
part-task  research.  Specifically,  the  fact  that  fidelity  defies 
simple  description  or  measurement  and  thus  levels  ^  of  required 
fidelity  can  not  be  determined  in  general  for  various  types  of 
part-task  simulation  research. 

Conclusions 

Simulation  fidelity  is  an  obscure  concept  that  is  being  thrust 
onto  the  simulation  community  as  a  way  to  measure  a  simulation 
device's  effectiveness  for  human  factors  research.  To  date,  no 
consensus  on  just  what  exactly  fidelity  is,  or  how  it  affects 
simulation  research  efforts  has  been  agreed  upon.  Also,  there  is 
even  more  discrepancy  regarding  how  to  determine  fidelity 
requirements  for  specific  types  of  human  factors  research.  This 
document  analyzed  the  latest  research  pertaining  to  simulation 
research  fidelity  requirements  and  made  general  recoinmendations 
as  to  the  requirements  necessary  when  performing  various  types  of 
human  factors  simulation  research. 

In  the  past  few  years  simulation  has  become  more  acceptable  for^ 
research  due  to  technology  advancements  enabling  nearly  realistic 
levels  of  simulator  fidelity.  This  trend  is  likely  to  increase, 
given  the  steady  improvements  in  all  aspects  of  simulation.  A 
well-designed  research  simulation  project  is  cost-effective  when 
compared  to  most  other  ways  of  achieving  the  same  objectives, 
such  as  flight  test.  However,  as  the  use  of  simulation  for 
research  increases,  more  specific  guidelines  and  requirements  for 
fidelity  are  necessary  to  ensure  that  the  simulation  devices  are 
being  used  effectively  to  meet  the  objectives  of  the  specified 
research . 

6 .  FUTURE  RESEARCH  AND  RECOMMENDATIONS . 

As  shown  in  this  analysis,  fidelity  is  an  ambiguous  concept  not 
clearly  defined  or  agreed  upon  in  the  simulation  research 
community.  The  benefits  of  better  understanding  fidelity  and  its 
requirements  for  simulation  research  are  numerous.  Specifically, 
the  ability  to  determine  the  exact  amount  of  fidelity  necessary 
for  an  individual  research  objective,  and  then  be  able  to  choose 
the  correct  simulation  device  for  the  desired  level  of  fidelity 
would  reduce  the  chance  of  conducting  research  with  an  inadequate 
level  of  simulation.  To  reiterate,  current  decisions  on 
simulation  devices  for  research  are  resulting  in  unnecessary 
costs  and  extraneous  factors  negatively  effecting  results  when 
too  much  fidelity  is  utilized,  and  behaviors  that  do  not 
represent  real-life  are  providing  insufficient  results  when  not 
enough  fidelity  is  incorporated. 

Presently,  there  are  no  guidelines  for  determining  fidelity 
requirements  for  various  simulation  research  applications . 
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Furthermore,  there  is  no  consensus  on  the  effects  varying  levels 
of  fidelity  have  on  research  results,  or  for  the  amount  of 
fidelity  necessary  to  elicit  useful  evaluations  of  behavior.  To 
provide  answers  to  these  questions  and  begin  to  have  a  better 
understanding  of  simulator  fidelity  in  its  entirety,  several 
topics  for  future  research  are  recommended. 

One  area  of  fidelity  research  that  needs  more  attention  is  the 
determination  of  a  quantitative  method  of  defining  fidelity.  As 
mentioned  earlier,  fidelity  is  a  metric-free  subjective _ value  _ 
perceived  differently  from  one  person  to  the  next.  Variations  in 
the  fidelity  of  individual  components,  as  well  as  the  simulator 
as  a  whole  need  to  have  a  universal  index  for  comparison.  This 
index  could  be  used  to  determine  a  specific  value  of  fidelity 
necessary  for  a  research  objective,  and  also  give  a  method  of 
comparing  one  simulator  to  another  based  on  level  of  fidelity. 

Further  research  is  also  needed  for  the  determination  of  specific 
criteria  for  classifying  simulators  in  terms  of  overall  fidelity. 
Currently  the  Federal  Aviation  Administration  (FAA)  categorizes 
aircraft  simulation  devices  in  terms  of  objective  (engineering) 
fidelity,  as  illustrated  in  Advisory  Circulars  No.  120-40B  and 
120-45A.  Simulation  devices,  for  research  especially ,  need 
categorized  in  terms  of  their  perceptual  fidelity  as  well.  This 
would  allow  a  researcher  with  specific  fidelity  requirements  to 
choose  a  simulator  that  is  appropriate.  Simulator  classification 
by  fidelity  sets  a  basis  from  which  the  user  coinmunity  can 
identify  the  specific  simulation  device  that  is  optimized  for 
their  needs  (Prasad  et  al . ,  1991). 

Also,  further  analysis  and  examination  of  human  factors 
simulation  studies  will  provide  guidance  for  fidelity  issues  that 
need  to  be  considered  in  future  simulation  studies.  Upon 
completion  of  a  research  study,  practitioners  could  analyze  the 
effects  that  the  level  of  fidelity  utilized  had  on  the  desired 
outcome.  Study  specifics  such  as  type  of  simulator,  measures 
evaluated,  fidelity  characteristics  and  a  general  description  of 
the  experimental  study  from  subject  and  observer  coiments  would 
be  obtained.  The  information  could  be  stored  in  a  database  and 
be  available  for  future  researchers  to  access  when  addressing 
f j_(3,elity  concerns  for  their  own  human  factors  simulation 
research. 

Lastly  a  process  by  which  to  determine  what  type  of  simulation 
research  to  conduct,  full-mission  or  part-task,  and  the  type  of 
simulation  device  to  utilize  for  a  specific  research  evaluation, 
is  needed  for  researchers  to  obtain  the  most  benefit  from 
research  conducted  using  simulation.  The  experimental  factors 
included  in  the  following  checklist  provide  a  list  of  issues  to 
be  examined  when  determining  the  type  of  experimental  simulation 
device,  and  level  of  sophistication,  for  a  particular  study. 

Upon  addressing  these  issues  of  concern  for  simulation  research, 
a  researcher  can  then  match  the  specified  needs  with  the 


23 


characteristics  and  capabilities  found  in  the  different  study- 
types  to  determine  which  will  result  in  the  most  appropriate 
evaluation  for  the  planned  research.  For  future  FAA  Technical 
Center  simulation  research,  it  is  recommended  that  this  process 
be  used  to  determine  simulator  sophistication  requirements.  A 
committee  of  experts  made  up  of  Crew  System  Ergonomics 
Information  Analysis  Center  (CSERIAC)  and  FAA  personnel  could 
objectively  determine  these  requirements  based  on  a  specific 
study's  characteristics,  as  determined  by  addressing  issues 
listed  in  the  checklist  below. 


Checklist  of  Experimental  Factors  to  Consider  for  Simulation 
_ Research _ 

_ GENERAL _ 

-  Time 

-  Cost 

-  Aircraft  (modeVseries ) 

-  Systems  to  be  tested 

-  Statistical  Power  needed 

-  External  Factors  of  Interest _ 

-  _  EXPERIMENTAL  METHODOLOGY  . _ ' 

-  Research  Objective 

-  Variables  of  Interest 

-  Behavior  of  Interest 

-  Physiological  Factors  to  be  stimulated 

-  Psychological  Factors  to  be  stimulated 

-  Procedures /Skills  to  be  tested 


The  use  of  simulators  for  research  is  increasing  as  technology 
enhancements  improve  their  capabilities.  A  better  understanding 
of  the  concept  of  fidelity  is  necessary  to  assure  that  simulation 
devices  are  utilized  correctly  for  human  factors  research.  The 
recommendations  presented  herein  for  simulator  fidelity  issues  to 
address  in  the  future  will  start  providing  answers  to ' the 
questions  and  concerns  expressed  throughout  this  document. 
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APPENDIX  A 


FLIGHT  TRAINING  DEVICE  MINIMUM  STANDARDS 
AS  SPECIFIED  IN  FAA-AC- 120-45 A 


Table  A  Flight  Training  Device  Minimum  Standards 


LEVEL 


GENERAL  _ _ 


a.  A  cockpit  which  will  have  actuation 
of  controls  and  switches  which  replicate 
those  in  the  airplane. 


b.  Instruments,  equipment,  panels, 
systems,  and  controls  sufficient  for  the 
training/checking  events  to  be 
accomplished  must  be  located  in  a 
spatially  correct  open  flight  deck  area. 
Actuation  of  these  controls  and  switches 
must  reolicate  those  in  the  airplane, _ 


c.  Daily  preflight  documentation. 


d.  Lighting  environment  for  panels 
and  instruments  must  be  sufficient  for  the 
operation  being  conducted.  _ _ 


e.  Circuit  breakers  should  function 
accurately  when  they  are  involved  in 
operating  procedures  or  malfunctions  re- 
uiring  or  involving  flight  crew  response. 


f  Effect  of  aerodynamic  changes  for 
various  combinations  of  drag  and  thrust 
normally  encountered  in  flight,  including 
the  effect  of  change  in  airplane  attitude, 
thrust,  drag,  altitude,  temperature,  and 
configuration  _ 


g.  Digital  or  analog  computing  of 

sufficient  capacity  to  conduct  complete 
operation  of  the  device  including  its 
evaluation  and  testing. _ _ 

h.  All  relevant  instrument  indications 

involved  in  the  simulation  of  the 
applicable  airplane  entirely  automatic  in 
response  to  control  input. _ _ 


i.  Navigation  equipment 
corresponding  to  that  installed  in  the 
replicated  airplane  with  operation  within 
the  tolerances  prescribed  for  the  actual 
airborne  equipment. 


Comments  _ 


Level  3  must  be  representative  of  a 
single  set  of  airplanes,  and  must  have 
navigation  controls,  displays,  and 
instrumentation  as  set  out  in  FAR 
Section  91.33  for  operation  in  accord¬ 
ance  with  instrument  flight  rules 


Level  2  must  be  representative  of  a 
single  set  of  airplanes.  Levels  2  and  5 
require  simulated  aerodynamic 
capability  and  control  forces  and  travel 
sufficient  to  manually  fly  an  instrument 
approach. 


X  X  X  X  X  X  Lighting  must  be  as  per  airplane 
lighting  for  Level  7. 


X  X  X  X  X  X  Must  be  properly  located  in  Levels  6 
and  7. 


Levels  3,  6,  and  7  require  additionally, 
the  effects  of  gross  weight  and  center  of 
gravity. 


Levels  3, 6,  and  7  must  also  include 
communication  equipment  (interphone 
and  air/ground)  corresponding  to  that 
installed  in  the  replicated  aircraft,  and, 
if  appropriate,  to  the  operation  being 
conducted,  an  oxygen  mask 
microphone/communication  system. 
Levels  2  and  5  need  have  operational 
only  that  navigation  equipment 
sufficient  to  fly  a  non-precision 
instrument  approach. 
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Table  A.  Flight  Training  Device  Minimum  Standards  ('cont’d') 


GENERAL 

LEVELS 

n 

B 

a 

a 

B 

B 

B 

Comments 

j.  Crewmember  seats  must  afford  the 
capability  for  the  occupant  to  be  able  to 
achieve  the  design  eye  reference  position 
for  specific  airplanes,  or  to  approximate 
such  a  position  for  a  generic  set  of 
airplanes. 

X 

X 

X 

X 

Level  7  crewmember  seats  must 
accurately  simulate  those  installed  in 
the  airplane. 

k.  In  addition  to  the  flight  crew¬ 
member  stations,  suitable  seating  arrange¬ 
ments  for  an  instructor/check  airman  and 
FAA  inspector.  These  seats  must  provide 
adeouate  view  of  crewmember's  panel(s). 

X 

X 

X 

X 

X 

X 

These  seats  need  not  be  a  replica  of  an 
aircraft  seat  and  can  be  as  simple  as  an 
office  chair  placed  in  an  appropriate 
position. 

1.  Installed  system(s)  must  simulate 
the  applicable  airplane  system  operation, 
both  on  the  ground  and  in  flight.  At  least 
one  airplane  system  must  be  represented. 
System(s)  must  be  operative  to  the  extent 
that  applicable  normal,  abnormal,  and 
emergency  operating  procedures  included 
in  the  operator's  training  programs  can  be 
accomplished. 

1 

X 

X 

X 

X 

X 

X 

Levels  6  and  7  must  simulate  all 
applicable  airplane  flight,  navigation, 
and  s>'stems  operation.  Level  3  must 
have  flight  and  navigational  controls, 
displays,  and  instrumentation  for 
powered  aircraft  as  set  out  in  FAR 

Section  91.33  for  IFR  operation. 

Levels  2  and  5  must  have  functional 
flight  and  navigational  controls, 
displays,  and  instrumentation. 

m.  Instructor  controls  that  permit 
activation  of  normal,  abnormal,  and 
emergency  conditions,  as  may  be 
appropriate.  Once  activated,  proper 
system  operation  must  result  from  system 
management  by  the  crew  and  not  require 
input  from  the  instructor  controls. 

X 

X 

X 

X 

X 

X 

n.  Control  forces  and  control  travel 
which  correspond  to  that  of  the  replicated 
airplane,  or  set  of  airplanes.  Control 
forces  should  react  in  the  same  manner  as 
in  the  airplane,  or  set  of  airplanes,  under 
the  same  flight  conditions. 

1 

X 

X 

X 

X 

X 

Levels  2  and  5  need  control  forces  and 
control  travel  only  of  sufficient 
precision  to  manually  fly  an 
instrument  approach. 

0.  Significant  cockpit  sounds  which 
result  from  pilot  actions  corresponding  to 
those  of  the  airplane. 

■ 

1 

X 

1 

1 

X 

X 

p.  Sound  of  precipitation,  windshield 
wipers,  and  other  significant  airplane 
noises  precipitable  to  the  pilot  during 
normal,  abnormal,  or  emergency 
operations,  as  may  be  appropriate. 

1 

1 

1 

1 

1 

1 

X 

Statement  of  Compliance. 
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Table  A.  Flight  Training  Device  Minimum  Standards  Ccont’d 


LEVELS 


GENERAL 


q.  Aerodynamic  modeling  which,  for 
airplanes  issued  an  original  type 
certificate  after  June  1980,  includes  low- 
altitude  level-flight  ground  effect,  Mach 
effect  at  high  altitude,  effects  of  airframe 
icing,  nonnal  dynamic  thrust  effect  on 
control  surfaces,  aeroelastic 
representations,  and  representations  of 
nonlinearities  due  to  sideslip  based  on 
airplane  flight  test  data  provided  by  the 
manufacturer. 


r.  Control  feel  dynamics  which 
replicate  the  airplane  simulated.  Initial 
and  upgrade  evaluation  will  include 
control  free  response  (column,  wheel,  and 
pedal)  measurements  recorded  at  the 
controls.  The  measured  responses  must 
correspond  to  those  of  the  airplane  in 
takeoff,  cruise  and  landing  configurations, 

(1)  For  airplanes  with  irreversible  control 
systems,  measurements  may  be  obtained 
on  the  ground  if  proper  pilot  static  inputs 
are  provided  to  represent  conditions 
typical  of  those  encountered  in  flight. 
Engineering  validation  or  airplane 
manufacturer  rationale  will  be  submitted 
as  justification  to  ground  test  or  omit  a 
configuration. 

(2)  For  flight  training  devices  requiring 
static  and  dynamic  tests  at  the  controls, 
special  test  fixtures  will  not  be  required 
during  initial  evaluations  if  the  operator's 
ATG  shows  both  test  fixture  results  and 
alternate  test  method  results,  such  as 
computer  data  plots,  which  were  obtained 
concurrently.  Repeat  of  the  alternate 
method  during  the  initial  evaluation  may 
then  satisfy  this  test  requirement. 


s.  Aerodynamic  and  ground  reaction 
modeling  for  the  effects  of  reverse  thrust 
on  directional  control. 


t.  Timely  permanent  update  of  flight 
training  device  hardware  and 
programming  consistent  with  airplane 
modifications. 


Comments 


Statement  of  Compliance. 

Tests  required.  The  statement  must 
address  ground  effect,  Mach  effect, 
aeroelastic  representations,  and 
nonlinearities  due  to  sideslip. 
Separate  tests  for  thrust  effects  and 
demonstration  of  icing  effects  are 
required. 


X  Statement  of  Compliance. 


X  Statement  of  Compliance. 
Tests  required. 


X  X  X  X  X  X 
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Table  A.  Flight  Training  Device  Minimum  Standards  (cont’d) 


GENERAL 

LEVELS 

n 

a 

il 

a 

a 

a 

a 

Comments 

u.  Visual  system;  if  installed  (not 
required). 

1 

X 

X 

X 

X 

X 

X 

Visual  sj’stem  standards  set  out  in  AC 
120-40,  as  amended,  for  at  least  Level 

A  simulators  will  be  acceptable. 

V.  Motion  system;  if  installed  (not 
required). 

1 

X 

X 

X 

X 

X 

X 

Motion  s>’stem  standards  set  out  in  AC 
120-40,  as  amended,  for  at  least  Level 

A  simulators  will  be  acceptable. 
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